Style transfer and image generation are fun but are ends unto themselves if you want to use them as artistic tools--this looks like it could be deployed and replace basic character locomotion animation.

Regular reality is being disrupted by virtual and augmented reality (VR/AR). The biggest names in tech are battling to power the next generation of entertainment, education, and communication. Facebook acquired Oculus to make next generation social networking virtual. Apple CEO Tim Cook claims augmented reality will be "as big as the iPhone." Microsoft's HoloLens, Google's Tango, and Intel's Project Alloy are just a few of the myriad of developments underway to make VR/AR devices as ubiquitous as computers and phones.

We present a real-time character control mechanism using a novel neural network architecture called a Phase-Functioned Neural Network. In this network structure, the weights are computed via a cyclic function which uses the phase as an input. Along with the phase, our system takes as input user controls, the previous state of the character, the geometry of the scene, and automatically produces high quality motions that achieve the desired user control. The entire network is trained in an end-to-end fashion on a large dataset composed of locomotion such as walking, running, jumping, and climbing movements fitted into virtual environments. Our system can therefore automatically produce motions where the character adapts to different geometric environments such as walking and running over rough terrain, climbing over large rocks, jumping over obstacles, and crouching under low ceilings.

This year at SIGGRAPH I am presenting Phase-Functioned Neural Networks for Character Control. This paper uses a new kind of neural network called a "Phase-Functioned Neural Network" to create a character controller suitable for games. Our controller requires very little memory, is fast to compute at runtime, and generates high quality motion in many complex situations. We also present a technique for fitting terrains from virtual environments to separately captured motion data. This is used to train our system so it can natually traverse rough terrains at runtime.